Employing Textual and Facial Emotion Recognition to Design an Affective Tutoring System

Emotional expression in Artificial Intelligence has gained lots of attention in recent years, people applied its affective computing not only in enhancing and realizing the interaction between computers and human, it also makes computer more humane. In this study, emotional expressions were applied into intelligent tutoring system, where learners’ emotional expression in learning process was observed in order to give an appropriate feedback. Emotional intelligent not only gives high flexibility to the interaction of tutoring system, it also to deepen its level of human interaction. This study uses dual-mode operation: facial expression recognition, and text semantics as the main elements in affective computing to understand users’ emotions. Text semantics are used to understand learner’s learning status, and the results would contribute to course management agents in order to choose the most appropriate teaching strategies and feedback to the users. Facial expression recognition allows interactive agents to provide users a complete sound and animation feedback.. INTRODUCTION Affective computing means to obtain facial expressions and signals of physical change aroused by emotions and feelings via various sensors and to recognize these signals so as to understand human emotions to give proper feedback (Li & Cheng, 2011; Li & Huang, 2007; Liao et al., 2010; Manovich, 2001). This is an emerging academic field, concerning the study of detecting human emotions and establishing proper emotional models so that emotions may be expressed in every possible way and even be transmitted on the Internet (MIT Media Lab, 2008). In this case, affective computing is regarding two aspects: affection and emotion. Therefore, it will also need to detect information from both physical and psychological resources. According to Ammar et al. (2010), the latest scientific study has proved that emotions do exert great influence on decision making, perception, and learning. For the time being, most “intelligent tutoring systems (ITS)” place more emphasis on providing users with a highly flexible and interactive learning environment. Moreover, they may present users with proper learning materials along with teaching strategies based on their background knowledge. For example, when users do not reach desired grades, the tutoring system may lower the learning level timely to suit the users’ needs. Nevertheless, studies concerning the learning status of users are rare to be found. For instance, when a user’s learning capability is in decline, the cause for his/her weakened learning motivation may come from the individual’s mood swings rather than his/her poor capability of learning. In this case, it is hoped that the involvement of affective computing may help to observe users’ emotions and learning status so that the fluctuation may help the tutoring system to provide users with suitable courses and feedback (Lin et al., 2011a; Lin et al., 2011b; Tsai et al., 2010). However, humans have a complicated way to express their emotions, such as facial expression, eye contact, body language, physiological phenomenon, and even words. In this case, any single method is not likely to obtain affection or situation in a complete manner. Therefore, this study suggests adopting facial expression recognition and text semantics as a dual-mode operation so that information regarding a user’s emotions and learning status may be discovered and understood better (Willems, 2011; Abulibdeh & Hassan, 2011; Yeo & Que, 2011). This study is aimed at applying affective computing to ITS so that a user’s learning status as well as immediate emotions may be considered an index for the reference of flexible tutoring courses. TOJET: The Turkish Online Journal of Educational Technology – October 2012, volume 11 Issue 4